Image processing system and method for detecting a target object area based on height information

ABSTRACT

An image processing system, an imager, an area detection method, and a computer program easily and highly accurately detecting an area where a target object exists in an image acquired by imaging the target object are provided. The image processing system includes a projector for projecting a projection image having a stripe pattern or a grid pattern toward a target object, and an imager for imaging a target object on which the projection image is projected, and the imager includes an input image obtaining module for obtaining an input image acquired by imaging a target object on which the projection image is projected, a height information calculator for calculating height information of each pixel in the input image by using the input image, and a target object area detector for detecting a target object area where the target object exists in the input image, based on the height information of each pixel in the input image.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a National Phase Patent Application and claimspriority to and the benefit of International Application NumberPCT/JP2015/066778, filed on Jun. 10, 2015, the entire disclosure ofwhich is incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to an image processing system, an imager,an area detection method, and a computer program, and more particularly,the present disclosure relates to an image processing system, an imager,an area detection method, and a computer program for detecting an areawhere a target object exists from an input image.

BACKGROUND

In recent years, when a product having a three-dimensional shape isoffered for sale on an electronic commerce (EC) site, an auction site,etc., a three-dimensional image is posted as a product image, so thatviewers of the site can easily understand the shape of the product. Theseller of the product can create a three-dimensional image of theproduct, for example, by using a projector that projects a predeterminedprojection image and a camera that captures the image of the product onwhich the projected image is projected.

An image input apparatus that causes a light projecting module to emit apredetermined light projection pattern on a subject and causes animaging module to capture a projected image having a distortion of theprojected pattern. In this image input apparatus, the relative positionbetween the light projection module and the imaging module is fixed, andthe moving module relatively moves the imaging module and accordinglythe imaging module captures images of multiple light projection imagesat different imaging positions is disclosed (see Patent Literature 1).

An object identification apparatus, which captures an image of a breadand a tray while backlight is emitted from the back of the tray carryingthe bread, crops the bread area from the captured color digital image,and identifies the type of the bread based on the cropped bread area isdisclosed (see Patent Literature 2).

CITATION LIST Patent Literature

Patent Literature 1: Japanese Unexamined Patent Publication (Kokai) No.2003-78725

Patent Literature 2: Japanese Unexamined Patent Publication (Kokai) No2013-145441

SUMMARY Technical Problem

When a three-dimensional image of the target object is generated from animage acquired by imaging the target object, it is preferred to easilyand highly accurately detect the area where the target object exists.

It is an object of the image processing system, the imager, the areadetection method, and the computer program to easily and highlyaccurately detect the area where the target object exists in the imageacquired by imaging the target object.

Solution Problem

The image processing system according to an embodiment includes aprojector for projecting a projection image having a stripe pattern or agrid pattern toward a target object, and an imager for imaging a targetobject on which the projection image is projected, the imager includesan input image obtaining module for obtaining an input image acquired byimaging the target object on which the projection image is projected, aheight information calculator for calculating height information of eachpixel in the input image by using the input image, and a target objectarea detector for detecting a target object area where the target objectexists in the input image, based on the height information of each pixelin the input image.

The imager according to an embodiment includes an input image obtainingmodule for obtaining an input image acquired by imaging a target objecton which a projection image having a stripe pattern or a grid pattern isprojected by a projector, and a height information calculator forcalculating height information of each pixel in the input image by usingthe input image, and a target object area detector for detecting atarget object area where the target object exists in the input image,based on the height information of each pixel in the input image.

The area detection method according to an embodiment is an areadetection method for an image processing system including a projectorand an imager, the area detection method includes projecting, by theprojector, a projection image having a stripe pattern or a grid patterntoward a target object, obtaining, by the imager, an input imageacquired by imaging the target object on which the projection image isprojected, calculating, by the imager, height information of each pixelin the input image by using the input image, detecting, by the imager, atarget object area where the target object exists in the input image,based on the height information of each pixel in the input image.

The computer program according to an embodiment causes a computer toexecute obtaining an input image acquired by imaging a target object onwhich a projection image having a stripe pattern or a grid pattern isprojected by a projector, calculating height information of each pixelin the input image by using the input image, and detecting a targetobject area where the target object exists in the input image, based onthe height information of each pixel in the input image.

Advantageous Effects of Invention

According to the present embodiment, the image processing system, theimager, the area detection method, and the computer program can easilyand highly accurately detect the area where the target object exists inthe image acquired by imaging the target object.

The object and advantages of the invention will be realized and attainedby means of the elements and combinations, in particular, pointed out inthe claims. It is to be understood that both the foregoing generaldescription and the following detailed description are exemplary andexplanatory, and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a diagram illustrating a schematic configuration of an imageprocessing system 1 according to an embodiment.

FIG. 1B is an exploded view illustrating the image processing system 1according to the embodiment.

FIG. 2 is a hardware configuration diagram illustrating an imager 100.

FIG. 3 is a diagram illustrating a schematic configuration of a firstCPU 110.

FIG. 4 is a hardware configuration diagram illustrating a projector 200.

FIG. 5 is a flowchart illustrating operation of area croppingprocessing.

FIG. 6 is a flowchart illustrating an example of operation of heightinformation calculation processing.

FIG. 7A is a schematic diagram explaining each input image.

FIG. 7B is a schematic diagram explaining each input image.

FIG. 7C is a schematic diagram explaining each input image.

FIG. 7D is a schematic diagram explaining each input image.

FIG. 8 is a schematic diagram for explaining a calculation method ofdepth.

FIG. 9 is a flowchart illustrating an example of operation of heightinformation calculation processing.

FIG. 10 is a schematic diagram for explaining an estimated backgroundarea.

FIG. 11 is a flowchart illustrating an example of operation of shadowarea detection processing.

FIG. 12A is a schematic diagram for explaining a first color inputimage.

FIG. 12B is a schematic diagram for explaining a second color inputimage.

FIG. 13 is a flowchart illustrating an example of operation of targetobject area detection processing.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, an image processing system, an imager, an area detectionmethod, and a computer program according to an embodiment, will bedescribed with reference to the drawings. However, note that thetechnical scope of the invention is not limited to these embodiments andextends to the inventions described in the claims and their equivalents.

FIG. 1A is a diagram illustrating a schematic configuration of an imageprocessing system according to an embodiment. FIG. 1B is an explodedview illustrating the image processing system. As illustrated in FIGS.1A, 1B, the image processing system 1 includes an imager (image readingapparatus) 100, a projector (image projection apparatus) 200, a platentable 300, an installation stand 400, and a support member 500, etc.

The imager 100 is an apparatus that is disposed above a target objectplaced on the platen table 300 and that captures an image of the targetobject. The imager 100 is, for example, a multifunctional mobile phone(a smartphone), a tablet personal computer (PC), a portable informationterminal, etc., having a camera, etc. The target object imaged by theimager 100 is, for example, an object having a stereoscopicthree-dimensional shape.

The projector 200 is an apparatus that is disposed above the targetobject placed on the platen table 300 and projects a projection imagetoward a target object. For example, the projector 200 is a projector.

The platen table 300 is a table on which the target object to be imagedby the imager 100 is placed.

The installation stand 400 is a stand on which the imager 100 isinstalled. The installation stand 400 has a U-shape including a base401, a connection member 402, and an installation member 403. The base401 has a planar shape and is attached to a predetermined position 301of the platen table 300 with screws, etc. The base 401 may not beattached to the determined position 301 but may simply be placedthereon. The connection member 402 has a planar shape and extends upwardin the vertical direction from the base 401. The installation member 403has a planar shape, and is attached to the vertically upper end of theconnection member 402 in the same direction as the base 401 is attached.The imager 100 is placed on the installation member 403 so that thetarget object placed on the platen table 300 can be imaged, i.e., theimaging direction is vertically downward and the installation member 403is not included in the imaging range.

The support member 500 is a member that supports the projector 200. Thesupport member 500 has an L shape and is attached to the predeterminedposition 302 of the platen table 300 with screws, etc., and the supportmember 500 extends upward in the vertical direction from the platentable 300. The projector 200 is attached to the end of the supportmember 500 at the side opposite to the end attached to the platen table300 with screws etc. The projector 200 is attached to the support member500 so that the projection image can be projected on the target objectplaced on the platen table 300, i.e., the projection direction isvertically downward.

As illustrated in FIG. 1A and FIG. 1B, the installation stand 400 andsupport member 500 are arranged at predetermined positions 301 and 302,respectively, of the platen table 300, so that the arrangement positionof the imager 100, the projector 200, and the platen table 300 arefixed. As a result, the position relation is maintained to be constantbetween the optical center and imaging direction of the imagingapparatus of the imager 100, the optical center and projection directionof the projector 200, and the placing plane of the platen table 300.

The support member 500 may be attached to the installation stand 400rather than being attached to the platen table 300. The imager 100 andthe installation stand 400 may be integrally configured using, forexample, an overhead scanner apparatus, etc.

FIG. 2 is a hardware configuration diagram illustrating the imager 100.As illustrated in FIG. 2, the imager 100 includes a first communicationcircuit 101, an imaging apparatus 102, an input apparatus 103, a displayapparatus 104, a first storage apparatus 105, a first central processingunit (CPU) 110, etc. Each module of the imager 100 will be describeddetails below.

The first communication circuit 101 includes an antenna for transmittingand receiving radio signals and a wireless communication interfacecircuit for transmitting and receiving a signal through a wirelesscommunication circuit according to a predetermined communicationprotocol such as a wireless local area network (LAN). The firstcommunication circuit 101 communicates with the projector 200 through anaccess point (not illustrated), transmits the information output fromthe first CPU 110 to the projector 200, and transmits the informationreceived from the projector 200 to the first CPU 110. It should be notedthat the first communication circuit 101 may directly communicate withthe projector 200 without relying on the access points. In addition, thefirst communication circuit 101 may communicate with the projector 200with short-range wireless communication.

The imaging apparatus 102 has an imaging sensor for imaging the targetobject. This imaging sensor includes an imaging device and an opticalsystem for forming the image of the target object on the imaging device.Each imaging device outputs analog values corresponding to RGB colors.The imaging device is a charge coupled device (CCD), a complementarymetal oxide semiconductor (CMOS), etc., arranged in one or arranged twodimensionally. The imaging apparatus 102 converts each analog valueoutput from the imaging sensor into a digital value to generate pixeldata, and generates image data (hereinafter referred to as input image)composed of each generated pixel data. In this input image, for example,each pixel data is color image data composed of a total of 24 bits ofRGB values represented with 8 bits for each color of RGB. The imagingapparatus 102 outputs the generated input image to the first CPU 110.

The input apparatus 103 has an input device such as a touch pad and aninterface circuit for obtaining a signal from the input device, andoutputs a signal corresponding to the operation of the user to the firstCPU 110.

The display apparatus 104 includes a display composed of liquid crystal,organic electro-luminescence (EL), etc., and an interface circuit thatoutputs image data to a display, and displays image data output from thefirst CPU 110.

The first storage apparatus 105 includes a volatile semiconductor memorysuch as a random access memory (RAM), a nonvolatile semiconductor memorysuch as a read only memory (ROM), etc. The first storage apparatus 105stores computer programs, databases, tables, various images, etc., usedfor various processing of the imager 100. The computer program may beinstalled on the first storage apparatus 105 from a computer-readable,non-transitory medium such as a compact disk read only memory (CD-ROM),a digital versatile disk read only memory (DVD-ROM), or the like byusing a well-known setup program or the like.

The first storage apparatus 105 stores a positional relationship betweenthe optical center of the imaging apparatus 102 and the optical centerof the projector 200, a positional relationship between the opticalcenter of the imaging apparatus 102 and the placing plane of platentable 300, etc. For example, a vector going from the optical center ofthe imaging apparatus 102 to the optical center of the projector 200 isstored as a positional relationship between the optical center of theimaging apparatus 102 and the optical center of the projector 200. Thedistance from the optical center of the imaging apparatus 102 to theplacing plane of the platen table 300, the angle of the imagingdirection of the imaging apparatus 102 with respect to the placing planeof the platen table 300, etc., are stored as the positional relationshipbetween the optical center of the imaging apparatus 102 and the placingplane of the platen table 300. These pieces of information are measuredin advance at the time of product shipment or installation of the imageprocessing system 1, etc.

The first CPU 110 is connected to the first communication circuit 101,the imaging apparatus 102, the input apparatus 103, the displayapparatus 104, and the first storage apparatus 105, and controls thesemodules. The first CPU 110 performs data transmission and receptioncontrol for transmission to and reception from the projector 200 via thefirst communication circuit 101, image generation control of the imagingapparatus 102, input control of the input apparatus 103, display controlof the display apparatus 104, control of the first storage apparatus105, etc.

FIG. 3 is a diagram illustrating a schematic configuration of the firstCPU 110. As illustrated in FIG. 3, the first CPU 110 includes aprojection control module 111, an input image obtaining module 112, aheight information calculator 113, an estimated background areaextractor 114, a color component extractor 115, a shadow area detector116, a target object area detector 117, a cropping module 118, etc. Eachof these modules is a functional module implemented with softwarerunning on a processor. Alternatively or in addition, each of thesemodules may be composed of integrated circuits, microprocessors,firmware, etc., independent from each other.

FIG. 4 is a hardware configuration diagram illustrating the projector200. As illustrated in FIG. 4, the projector 200 includes a secondcommunication circuit 201, a projection apparatus 202, a second storageapparatus 203, a second CPU 210, etc. Each module of the projector 200will be described details below.

The second communication circuit 201 has an interface circuit similar tothe first communication circuit 101 of the imager 100. The secondcommunication circuit 201 communicates with the imager 100, transmitsinformation output from the second CPU 210 to the imager 100, andoutputs the information received from the imager 100 to the second CPU210.

The projection apparatus 202 includes a light source, an optical system,etc., and projects the image output from the second CPU 210 toward thetarget object.

The second storage apparatus 203 includes a volatile semiconductormemory such as a RAM, a nonvolatile semiconductor memory such as a ROM,etc. The second storage apparatus 203 stores computer programs,databases, tables, various images, etc., used for various processing ofthe projector 200. The computer program may be installed on the secondstorage apparatus 203 from a computer-readable, non-transitory mediumsuch as a CD-ROM, a DVD-ROM, or the like by using a well-known setupprogram or the like.

The second CPU 210 is connected to the second communication circuit 201,the projection apparatus 202, and the second storage apparatus 203, andcontrols these modules. The second CPU 210 performs data transmissionand reception control for transmission to and reception from the imager100 via the second communication circuit 201, image projection controlof the projection apparatus 202, control of the second storage apparatus203, etc.

FIG. 5 is a flowchart illustrating operation of area cropping processingperformed by the imager 100. Hereinafter, the operation of area croppingprocessing will be described with reference to the flowchart illustratedin FIG. 5. The flow of the operation to be described below is executedmainly by the first CPU 110 in cooperation with each element of theimager 100 based on the program stored in the first storage apparatus105 in advance.

First, the first CPU 110 performs height information calculationprocessing (step S101). In height information calculation processing,the first CPU 110 calculates the height information of each pixel in aninput image (which may be hereinafter referred to as “projection inputimage”) acquired by imaging a target object on which the projectionimage is projected. The projection input image is an example of an inputimage. The details of height information calculation processing will bedescribed later.

Subsequently, the first CPU 110 performs background color determinationprocessing (step S102). In the background color determinationprocessing, the first CPU 110 determines the color component of thebackground in an input image (which may be hereinafter referred to asnon-projection input image) acquired by imaging a target object on whichno projection image is projected. The non-projection input image is anexample of second input image. The details of the background colordetermination processing will be described later.

Subsequently, the first CPU 110 performs shadow area detectionprocessing (step S103). In the shadow area detection processing, thefirst CPU 110 detects a shadow area in which a shadow is illustrated inthe projection input image and the non-projection input image. Thedetails of shadow area detection processing will be described later.

Subsequently, the first CPU 110 performs the target area detectionprocessing (step S104). In the target area detection processing, thefirst CPU 110 detects the target object area in which the target objectexists in the projection input image and the non-projection input image.The details of the target object area detection processing will bedescribed later.

Subsequently, the first CPU 110 generates a cropping image by croppingthe detected target object area from the non-projection input image,stores the detected target object area in the first storage apparatus105 (step S105), and terminates the series of steps. The user can usethe input apparatus 103 to display, on the display apparatus 104, thecropped image stored in the first storage apparatus 105, or the user canuse the input apparatus 103 to send the cropped image to anotherapparatus. It should be noted that the first CPU 110 may convert thecropped image into a three-dimensional image using the heightinformation calculated in the height information calculation processing.In the three-dimensional image, height information is stored inassociation with the pixel value for each of the pixels arranged in twodimensions, and based on the height information of each pixel,three-dimensional image is stereoscopically displayed on the displayapparatus 104. When the three-dimensional image is displayed on thedisplay apparatus 104, the first CPU 110 switches the viewpoint of thedisplayed three-dimensional image according to an instruction given bythe user using the input apparatus 103.

FIG. 6 is a flowchart illustrating an example of operation of heightinformation calculation processing. The flow of the operationillustrated in FIG. 6 is executed in step S101 of the flowchartillustrated in FIG. 5.

The first CPU 110 performs three-dimensional measurement of the targetobject according to the phase shift method. In order to perform thethree-dimensional measurement, the projection control module 111 firsttransmits a first request signal for requesting to project the firstprojection image toward the target object to the projector 200 via thefirst communication circuit 101 (step S201). The first request signalincludes a first projection image. The first projection image is animage having a stripe pattern or grid pattern. It should be noted that,in the later-described embodiment, an example using an image having astripe pattern is illustrated as an example.

Upon receiving the first request signal via the second communicationcircuit 201, the second CPU 210 of the projector 200 stores the firstprojection image included in the first request signal in the secondstorage apparatus 203 and causes the projection apparatus 202 to projectthe first projection image. It should be noted that the first projectionimage may be stored in advance in the second storage apparatus 203 ofthe projector 200 and the first CPU 110 may transmit the first requestsignal including the identification information of the first projectionimage instead of the first projection image.

Subsequently, the imaging apparatus 102 generates a first projectioninput image acquired by imaging the target object on which the firstprojection image is projected, and outputs the first projection inputimage to the first CPU 110. The input image obtaining module 112 obtainsthe first projection input image, and stores the first projection inputimage in the first storage apparatus 105 (step S202).

FIG. 7A to FIG. 7D are schematic diagrams for explaining input imagesimaged by the imaging apparatus 102. The image 700 illustrated in FIG.7A is an example of a non-projection input image. As illustrated in FIG.7A, the non-projection input image 700 is an image acquired by imaging atea cup 701 placed on the platen table 300 as a target object. The teacup 701 has a plain portion 703 having a color similar to that of thebackground 702 in which the platen table 300 is illustrated and apattern 704 having a color different from that of the background 702.

The image 710 illustrated in FIG. 7B is an example of a first projectioninput image. As illustrated in FIG. 7B, the first projection input image710 is an image acquired by imaging a tea cup 701 on which apredetermined sinusoidal pattern is projected as a first projectionimage. The sinusoidal pattern has a stripe pattern in which a whiteportion 711 and a predetermined color portion 712 other than the whiteportion are periodically repeated. On the platen table 300 having aplanar shape, the stripe based on the sinusoidal pattern is a straightline. The stripe based on the sinusoidal pattern is distorted and doesnot become a straight line on the tea cup 701 having a bulging shapewith respect to platen table 300.

Subsequently, the projection control module 111 transmits a secondrequest signal for requesting to project the second projection imagetoward the target object to the projector 200 via the firstcommunication circuit 101 (step S203). The second request signalincludes the second projection image. The second projection image is animage having a stripe pattern or a grid pattern.

Upon receiving the second request signal via the second communicationcircuit 201, the second CPU 210 of the projector 200 stores the secondprojection image included in the second request signal into the secondstorage apparatus 203 and causes the projection apparatus 202 to projectthe second projection image.

Subsequently, the imaging apparatus 102 generates a second projectioninput image acquired by imaging the target object on which the secondprojection image is projected, and outputs the second projection inputimage to the first CPU 110. The input image obtaining module 112 obtainsthe second projection input image and stores the second projection inputimage in the first storage apparatus 105 (step S204).

The image 720 illustrated in FIG. 7C is an example of a secondprojection input image. As illustrated in FIG. 7C, the second projectioninput image 720 is an image acquired by imaging the tea cup 701 on whicha sinusoidal pattern acquired by shifting the phase by π/2 from thesinusoidal pattern of the first projection image is projected as thesecond projection image. In the second projection input image 720, thewhite portion 721, and the predetermined color portion 722 are projectedto the position where the phase is shifted by π/2 from the positionwhere the white portion 711 and the predetermined color portion 712 areprojected in the first projection input image 710.

Subsequently, the projection control module 111 transmits a thirdrequest signal for requesting to project the third projection imagetoward the target object to the projector 200 via the firstcommunication circuit 101 (step S205). The third request signal includesa third projection image. The third projection image is an image havinga stripe pattern or a grid pattern.

Upon receiving the third request signal via the second communicationcircuit 201, the second CPU 210 of the projector 200 stores the thirdprojection image included in the third request signal in the secondstorage apparatus 203 and causes the projection apparatus 202 to projectthe third projection image.

Subsequently, the imaging apparatus 102 generates a third projectioninput image acquired by imaging the target object on which the thirdprojection image is projected, and outputs the third projection inputimage to the first CPU 110. The input image obtaining module 112 obtainsthe third projection input image and stores the third projection inputimage in the first storage apparatus 105 (step S206).

The image 730 illustrated in FIG. 7D is an example of a third projectioninput image. As illustrated in FIG. 7C, the third projection input image730 is an image acquired by imaging a tea cup 701 on which a sinusoidalpattern acquired by further shifting the phase by 2π/3 from thesinusoidal pattern of the second projection image is projected as thethird projection image. In the third projection input image 730, thewhite portion 731 and the preceding color portion 732 are projected tothe position where the phase is shifted by 2π/3 from the position wherethe white portion 721 and the predetermined color portion 722 areprojected in the second projection input image 720.

Subsequently, the height information calculator 113 calculates the phasevalue of the sinusoidal pattern at the position of each pixel from theluminance value of each pixel in the first projection input image, thesecond projection input image, and the third projection input image(step S207).

First, the height information calculator 113 calculates the luminancevalue I (x, y) of the pixel positioned at the coordinate (x, y) in eachprojection input image according to the following equation (1).[Math 1]I(x,y)=0.30×R(x,y)+0.59×G(x,y)+0.11×B(x,y)  (1)where R, G, B are color values for RGB in each projection input image.

Subsequently, the height information calculator 113 calculates the phasevalue phase (x, y) of the sinusoidal pattern at the position of thecoordinate (x, y) in each projection input image according to thefollowing equation (2)

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 2} \rbrack & \; \\{{{phase}\;( {x,y} )} = {\tan^{- 1}\frac{\sqrt{3}( {{I_{3}( {x,y} )} - {I_{1}( {x,y} )}} )}{2\;( {{2{I_{2}( {x,y} )}} - {I_{1}( {x,y} )} - {I_{3}( {x,y} )}} )}}} & (2)\end{matrix}$where I₁ (x, y) is the luminance value of the pixel positioned atcoordinate (x, y) in the first projection input image, I₂ (x, y) is theluminance value of the pixel positioned at the coordinate (x, y) in thesecond projection input image, and I₃ (x, y) is the luminance value ofthe pixel positioned at the coordinate (x, y) in the third projectioninput image.

The reason why the equation (2) is derived will be described below. Inthe following description, I₁ (x, y), I₂ (x, y), and I₃ (x, y) areexpressed as I₁, I₂, and I₃, respectively, in order to simplify theequation. When the phase value corresponding to the coordinate (x, y) inthe second projection input image is θ, then I₁, I₂, I₃ can be expressedby the following equation (3).

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 3} \rbrack & \; \\{{I_{1} = {{a\;{\cos( {\theta + \frac{2\;\pi}{3}} )}} + b}}{I_{2} = {{a\;{\cos(\theta)}} + b}}{I_{3} = {{a\;{\cos( {\theta - \frac{2\;\pi}{3}} )}} + b}}} & (3)\end{matrix}$where “a” is the intensity of the projected light, and “b” is theintensity of the ambient light. Therefore, the following equation (4) issatisfied.

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 4} \rbrack & \; \\{{{I_{1} - I_{3}} = {{- 2}\; a\;{\sin(\theta)}{\sin( \frac{2\;\pi}{3} )}}}{{I_{2} - I_{1}} = {2\; a\;{\sin( {\theta + \frac{\pi}{3}} )}{\sin( \frac{\pi}{3} )}}}{{I_{3} - I_{2}} = {2\; a\;{\sin( {\theta - \frac{\pi}{3}} )}{\sin( \frac{\pi}{3} )}}}} & (4)\end{matrix}$

The following equation (5) is satisfied based on the equation (4).

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 5} \rbrack & \; \\\begin{matrix}{\frac{\sqrt{3}( {I_{3} - I_{1}} )}{2( {{2\; I_{2}} - I_{1} - I_{3}} )} = \frac{{- \sqrt{3}}( {I_{1} - I_{3}} )}{2\{ {( {I_{2} - I_{1}} ) - ( {I_{3} - I_{2}} )} \}}} \\{= \frac{\sqrt{3}{\sin(\theta)}{\sin( \frac{2\;\pi}{3} )}}{2\;{\sin( \frac{\pi}{3} )}( {{\sin( {\theta + \frac{\pi}{3}} )} - {\sin( {\theta - \frac{\pi}{3}} )}} )}} \\{= \frac{\sqrt{3}{\sin(\theta)}{\sin( \frac{2\;\pi}{3} )}}{2\;{\sin( \frac{\pi}{3} )}( {{\cos(\theta)}{\sin( \frac{\pi}{3} )}} )}} \\{= {\tan(\theta)}}\end{matrix} & (5)\end{matrix}$

The equation (2) is satisfied based on the equation (5).

Since the phase value calculated in the equation (2) is calculated usingthe arctangent function, the phase value is continuous only in the valuerange of −π to π and is discontinuous in each period of the sinusoidalpattern. Therefore, for each phase period of sinusoidal pattern, theheight information calculator 113 adds 2π to the phase value of eachpixel calculated by the equation (2) and performs phase connection.Hereinafter, the phase value acquired by performing phase connection iscalled an absolute phase value.

Subsequently, the height information calculator 113 calculates the depthfor each corresponding pixel in the first projection input image, thesecond projection input image, and the third projection input image(step S208). The depth is the distance from the optical center of theimaging apparatus 102 to the target object illustrated in that pixel.

As described above, the pixels on the projection input image whoseabsolute phase values are identical correspond to the same line in thesinusoidal pattern on the projection image. Therefore, the heightinformation calculator 113 connects the pixels on the projection inputimage having the same absolute phase value, and the height informationcalculator 113 associates a group of connected pixels with each line ofthe sinusoidal pattern on the projection image. Further, for each pixelincluded in each group, the height information calculator 113 calculatesthe distances from the pixels positioned at both ends of the group. Theheight information calculator 113 associates each pixel included in eachgroup with each pixel on the line of the sinusoidal pattern associatedwith the group based on the calculated distance. Then, the heightinformation calculator 113 calculates the depth for each pixel on theprojection input image based on the positional relationship of eachpixel on the projection input image with the associated pixel on theprojection image.

FIG. 8 is a schematic diagram for explaining a calculation method ofdepth. In FIG. 8, an object 801 is a target object, a point c 811 is theoptical center of the imaging apparatus 102, a plane 812 is an imagingplane of the projection apparatus 202, a point p 821 is an opticalcenter of the projection apparatus 202, and a plane 822 is a projectionplane of the projection apparatus 202. A point s′ 813 on the imagingplane 812 and a point s″ 823 on the projection plane 822 correspond to apoint s 802 on the target object 801 and are associated with each other.As described above, in the image processing system 1, the positionalrelationship between the optical center of the imaging apparatus 102 andthe optical center of the projection apparatus 202 is kept constant. Avector T_(c-p) pointing from the point c 811 of the optical center ofthe imaging apparatus 102 to the point p 821 of the optical center ofthe projection apparatus 202 is stored in the first storage apparatus105 in advance.

An angle θ_(c) formed by the vector pointing from the point c 811 to thepoint p 821 and the vector pointing from the point c 821 to the point s802 on the target object 801 is calculated according to the followingequation (6) from the relationship of the inner product.

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 6} \rbrack & \; \\{\theta_{c} = {\cos^{- 1}( \frac{T_{c - s^{\prime}} \cdot T_{c - p}}{{T_{c - s^{\prime}}}\;{T_{c - p}}} )}} & (6)\end{matrix}$where T_(c-s′) is a vector pointing from the point c 811 of the opticalcenter of the imaging apparatus 102 and the point s′ 813 on the imagingplane 812.

Similarly, an angle θ_(p) formed by a vector pointing from the point c811 to the point p 821 and a vector pointing from the point p 821 to thepoint s 802 is calculated according to the following equation (7).

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 7} \rbrack & \; \\{\theta_{p} = {\cos^{- 1}( \frac{{- T_{p - s^{''}}} \cdot T_{c - p}}{{T_{p - s^{''}}}\;{T_{c - p}}} )}} & (7)\end{matrix}$where T_(p-s″) is a vector pointing from the point p 821 of the opticalcenter of the projection apparatus 202 to the point s″ 823 on theprojection plane 822.

The depth of the point s′ 813 on the imaging plane 812 is expressed bythe absolute value of the vector T_(c-s) pointing from the point c 821of the optical center of the imaging apparatus 102 to the point p 811 ofthe optical center of the projection apparatus 202. This depth iscalculated according to the following equation (8) by applying thetriangulation principle to a triangle having the point c, the point p,and the point s as vertexes.

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 8} \rbrack & \; \\{{depth} = {{T_{c - s}} = \frac{{T_{c - p}}\sin\;\theta_{p}}{\sin( {\pi - \theta_{c} - \theta_{p}} )}}} & (8)\end{matrix}$

Subsequently, the height information calculator 113 calculates theheight information of each pixel in the first projection input image,the second projection input image, and the third projection input imagebased on the calculated depth (step S209).

The height information calculator 113 calculates the distance from theoptical center of the imaging apparatus 102 to the point s 802 on thetarget object 801 in a direction orthogonal to placing plane based onthe depth of the point s′ 813 on the imaging plane 812 and the angle inthe imaging direction of the imaging apparatus 102 with respect to theplacing plane of the platen table 300. Based on the calculated distanceand the distance from the optical center of the imaging apparatus 102 tothe placing plane of the platen table 300, the height informationcalculator 113 calculates the height from placing plane to the point s802 on the target object 801. Then, the height information calculator113 stores the height from the calculated placing plane to the firststorage apparatus 105 as the height information of the pixelcorresponding to the point s′ 813 on the imaging plane 812.

As described above, the height information calculator 113 calculates theheight information of each pixel using the first projection input image,the second projection input image, and the third projection input image.

FIG. 9 is a flowchart illustrating an example of operation of heightinformation calculation processing. The flow of the operationillustrated in FIG. 9 is executed in step S102 of the flowchartillustrated in FIG. 5.

First, the projection control module 111 transmits a stop request signalfor requesting to stop the projection of projection image to theprojector 200 via the first communication circuit 101 (step S301).

Upon receiving the stop request signal via the second communicationcircuit 201, the second CPU 210 of the projector 200 causes theprojection apparatus 202 to stop the projection of the projection image.

Subsequently, the imaging apparatus 102 generates a non-projection inputimage acquired by imaging the target object on which no projection imageis projected, and outputs the non-projection input image to the firstCPU 110. The input image obtaining module 112 obtains a non-projectioninput image and stores the non-projection input image in the firststorage apparatus 105 (step S302).

Subsequently, based on the height information of each pixel calculatedby the height information calculator 113, the estimated background areaextractor 114 extracts the estimated target object area where it isestimated that the target object is illustrated from the non-projectioninput image (step S303).

The estimated background area extractor 114 determines, for each pixelin the non-projection input image, whether or not the height indicatedby the height information of the pixel corresponding to each pixel ismore than a predetermined value. The predetermined value is, forexample, zero. In consideration of the influence of the noise or theerror in calculation of the height information, the predetermined valuemay be a value more than 0 (value corresponding to 1 mm, for example).The estimated background area extractor 114 extracts pixels whose heightindicated by the height information exceeds the predetermined value ascandidates of the pixel in which the target object is illustrated. Theestimated background area extractor 114 groups the extracted pixels bylabeling, and extracts the extracted pixels as the estimated targetobject area.

Subsequently, the estimated background area extractor 114 extracts fromthe non-projected input image the estimated background area in which itis estimated that the background is illustrated based on the extractedestimated target area (step S304).

The estimated background area extractor 114 extracts a circumscribedrectangle including all the extracted estimated target object areas, andextracts a rectangle with a predetermined length margin at the outsideof the extracted circumscribed rectangle. The estimated background areaextractor 114 extracts a band-like area having a predetermined width(for example, 5 pixels) surrounding the extracted rectangle, andadjacent to the rectangle as the estimated background area.

It should be noted that the estimated background area extractor 114 mayextract a band-like area immediately adjacent to the extractedcircumscribed rectangle as the estimated background area. Alternatively,the estimated background area extractor 114 may extract only the areasat the four corners of the band-like area surrounding the extractedrectangle as the estimated background area.

FIG. 10 is a schematic diagram for explaining an estimated backgroundarea. An image 1000 illustrated in FIG. 10 is an example of anon-projection input image. In the non-projection input image 1000 asillustrated in FIG. 10, an area 1001 is an estimated target object area,a rectangle 1002 is a circumscribed rectangle of the estimated targetobject area 1001, and a rectangle 1003 is a rectangle having a margin1004 for the rectangle 1002. The band-like area 1005 is the estimatedbackground area. The areas 1006 to 1009 at the four corners of theband-like area may be the estimated background area.

Subsequently, the color component extractor 115 determines the colorcomponent of the background in the non-projection input image based onthe pixel in the estimated background area (step S305), and terminatesthe series of steps.

The color component extractor 115 calculates the average value of thecolor values of the RGB colors for each pixel in the estimatedbackground area, and the color component extractor 115 determines eachof the calculated average values as the color component of thebackground.

The user who captures the image of the target object pays attention soas not to cast the light directly on the vicinity of the target objectso that the vicinity of the target object is not affected by thelighting, but the user tends not to pay such attention to an area awayfrom the target object. On the other hand, instead of adopting theentire area outside of the estimated target object area as the estimatedbackground area, the estimated background area extractor 114 adopts onlythe area in proximity to the estimated target object area as theestimated background area. Accordingly, the color component extractor115 can accurately calculate the color component of the background fromthe area not affected by disturbance such as illumination. Since thecolor component extractor 115 does not calculate the background colorcomponent from the entire area outside the target object candidate area,the processing load related to the color component calculation can bereduced. Further, the color component extractor 115 further reduces theprocessing load related to the color component calculation by settingonly the areas at the four corners of the band-like area as theestimated background area.

FIG. 11 is a flowchart illustrating an example of operation of shadowarea detection processing. The flow of the operation illustrated in FIG.11 is executed in step S103 of the flowchart illustrated in FIG. 5.

First, the shadow area detector 116 determines whether or not the colorcomponent of the background determined by the color component extractor115 is black color (step S401).

The black color means color in the range of absorbing light. The shadowarea detector 116 determines that the color component of the backgroundis black color when each color value of RGB determined as the colorcomponent of the background is 32 or less, and the shadow area detector116 determines that the color component of the background is not blackcolor, when any color value is more than 32.

When the shadow area detector 116 determines that the color component ofthe background is black color, the shadow area detector 116 determinesthat the background is not affected by the shadow, and terminates theseries of steps without detecting the shadow area.

On the other hand, when the shadow area detector 116 determines that thecolor component of the background is not black color, the projectioncontrol module 111 transmits a first color request signal for requestingprojection of the light of the first color toward the target object tothe projector 200 via the first communication circuit 101 (step S402).The first color is, for example, green color.

Upon receiving the first color request signal via the secondcommunication circuit 201, the second CPU 210 of the projector 200causes the projection apparatus 202 to project the light of the firstcolor toward the target object.

Subsequently, the imaging apparatus 102 generates a first color inputimage acquired by imaging the target object on which the light of thefirst color is projected, and outputs the first color input image to thefirst CPU 110. The input image obtaining module 112 obtains the firstcolor input image and stores the first color input image in the firststorage apparatus 105 (step S403).

FIG. 12A is a schematic diagram for explaining the first color inputimage. An image 1200 illustrated in FIG. 12A is an example of a firstcolor input image. The first color input image 1200 illustrated in FIG.12A is an image acquired by imaging a target object 1202 having a blackcolor pattern 1201. As illustrated in FIG. 12A, the target object 1202and background 1203 have a tone of the first color, but the black colorpattern 1201 absorbs light of the first color and does not have a toneof the first color. A shadow 1204 is generated around the target object1202 due to light of the first color. The shadow 1204 does not have atone of the first color and is a gray color lighter than the black colordue to natural light or ambient light.

Subsequently, the projection control module 111 transmits a second colorrequest signal for requesting the projection of light of the secondcolor toward the target object to the projector 200 via the firstcommunication circuit 101 (step S404). The second color is a colordifferent from first color, for example pink color.

Upon receiving the second color request signal via the secondcommunication circuit 201, the second CPU 210 of the projector 200causes the projection apparatus 202 to project light of the second colortoward the target object.

Subsequently, the imaging apparatus 102 generates a second color inputimage acquired by imaging the target object on which light of the secondcolor is projected, and outputs the second color input image to thefirst CPU 110. The input image obtaining module 112 obtains the secondcolor input image and stores the second color input image in the firststorage apparatus 105 (step S405).

FIG. 12B is a schematic diagram for explaining the second color inputimage. An image 1210 illustrated in FIG. 12B is an example of the secondcolor input image. The second color input image 1210 illustrated in FIG.12B is an image acquired by imaging the same target object 1212 as thetarget object 1202 illustrated in FIG. 12A. As illustrated in FIG. 12B,the target object 1212 and background 1213 have the tone of the secondcolor, but the pattern 1211 of the black color absorbs the light of thesecond color and does not have the tone of the second color. The shadow1214 is generated by the light of the second color around the targetobject 1212. The shadow 1214 does not have the second color and is agray color lighter than black color due to natural light or ambientlight just like the shadow 1204 illustrated in FIG. 12A.

The processing in the following steps S406 to S410 is executed for eachof the pixel combinations existing in the corresponding positions in thefirst color input image and the second color input image.

First, the shadow area detector 116 determines whether or not thedifference in the color component between the pixel in the first colorinput image and the pixel in the second color input image correspondingto that pixel is equal to or less than a threshold value (step S406).The shadow area detector 116 calculates the difference between the colorvalues of two pixels for each color of RGB and calculates the sum of thecalculated differences as a difference of the color component. Thethreshold value is set to, for example, a color difference (for example,20) with which a person can visually discriminate the difference incolor on the image.

When the difference in the color component of each pixel is larger thana threshold value, the shadow area detector 116 determines that eachpixel has a first color tone and a second tone, and that the shadow isnot illustrated in that pixel (Step S407).

On the other hand, when the difference in the color component of eachpixel is equal to or less than the threshold value, the shadow areadetector 116 determines whether or not the color component of each pixelin the first color input image and the second color input image is blackcolor (step S408). The shadow area detector 116 determines whether ornot the color component of each pixel is black color just like stepS401.

When the color component of any pixel is black color, the shadow areadetector 116 determines that the target object or the background of theblack color is illustrated in the pixel, and that the shadow is notillustrated in the pixel (step S407).

On the other hand, when the color components of both pixels are not theblack color, the shadow area detector 116 determines that the pixelshave gray color, and that a shadow is illustrated in that pixel. Then,the shadow area detector 116 detects the position in each of theprojection input image and the non-projection input image correspondingto the position of the pixel as a shadow area (step S409).

Subsequently, the shadow area detector 116 determines whether or notprocessing has been executed for all the pixel combinations existing atthe corresponding positions in the first color input image and thesecond color input image (step S410). When the shadow area detector 116determines that there are pixel combinations for which the processinghas not yet been executed, the processing returns back to step S406 torepeat the processing in step S406 to S410. The shadow area detector 116terminates the series of steps when the processing is executed for allpixel combinations.

As described above, the shadow area detector 116 detects, as the shadowarea, the position in each of the projection input image and thenon-projection input image corresponding to the positions of the pixelcombinations in which the difference in the color components of thepixels is equal to or less than the threshold value and the pixel is notthe black color.

FIG. 13 is a flowchart illustrating an example of operation of targetobject area detection processing. The flow of the operation illustratedin FIG. 13 is executed in step S104 of the flowchart illustrated in FIG.5.

The processing in the following steps S501 to S506 is executed for eachof pixel combinations existing at the corresponding positions in thefirst to third projection input images, the non-projection input image,and the first to second color input images.

First, the target object area detector 117 determines whether or not ashadow is determined to be illustrated in the pixel of interest in thefirst color input image and the second color input image by the shadowarea detector 116 (step S501).

When the shadow is determined to be illustrated in the pixel ofinterest, the target object area detector 117 determines that the targetobject is not illustrated in the pixel (step S502).

On the other hand, when the shadow is determined not to be illustratedin the pixel of interest, the target object area detector 117 determineswhether or not the height indicated by the height information of thepixel in the projection input image corresponding to that pixel is morethan the predetermined value (step S503). The predetermined value is thesame value as the predetermined value used to extract the target objectcandidate area. It should be noted that the predetermined value may be avalue different from the predetermined value used to extract the targetobject candidate area.

When the height indicated by the height information of the pixel is morethan the predetermined value, the target object area detector 117determines that the target object is illustrated in the pixel (stepS504).

On the other hand, when the height indicated by the height informationof the pixel is equal to or less than the predetermined value, the colorcomponent extractor 115 extracts the color component of the pixel in thenon-projection input image corresponding to that pixel. Then, the colorcomponent extractor 115 determines whether or not the difference betweenthe extracted color component and the background color component isequal to or more than a predetermined value (step S505).

For each color of RGB, the color component extractor 115 calculates thedifference between the extracted color value and the color value of thebackground, and calculates the sum of the calculated differences as thedifference in the color component. The predetermined value is set to,for example, a color difference (for example, 20) with which a personcan visually discriminate the difference in color on the image.

When the difference between the extracted color component and thebackground color component is equal to or more than the predeterminedvalue, the target object area detector 117 determines that the targetobject is illustrated in the pixel (step S504).

On the other hand, when the difference between the extracted colorcomponent and the background color component is less than thepredetermined value, the target object area detector 117 determines thatthe target object is not illustrated in the pixel (step S505).

Subsequently, the target object area detector 117 determines whether theprocessing has been executed for all the pixels (step S506). When thereis a pixel for which the processing has not yet been executed, thetarget object area detector 117 returns the processing back to step S501to repeat the processing in step S501 to S506.

On the other hand, when the processing is executed for all the pixels,the target object area detector 117 generates a mask image foridentifying the area in which the target object is illustrated and thearea in which the target object is not illustrated (step S507). Thetarget object area detector 117 generates a mask image in which a pixelcorresponding to a pixel determined to illustrate the target object is avalid pixel, and a pixel corresponding to a pixel determined not toillustrate the target object is a invalid pixel.

Subsequently, the target object area detector 117 groups valid pixels inthe mask image by labeling (step S508).

Subsequently, the target object area detector 117 detects, as a targetobject area, the pixels at the positions corresponding to the positionswhere a group of valid pixels exists in the mask image in each of theprojection input image and the non-projection input image (step S509),and terminates the series of steps.

As described above, the target object area detector 117 detects thetarget object area in each projection input image based on the heightinformation of each pixel in each projection input image. As a result,as illustrated in FIG. 7A to FIG. 7D, the target object area detector117 can detect the target object area with a high degree of accuracyeven if the target object 701 has a color similar to the background.

The target object area detector 117 detects the target object area inthe non-projection input image based on the color component of eachpixel in the non-projection input image and the color component of thebackground. As a result, the target object area detector 117 can detectthe target object area with a high degree of accuracy based on the colorcomponent even if the target object has a planar shape with a lowheight.

The target object area detector 117 detects the target object area so asnot to include the shadow area detected by the shadow area detector 116.As a result, the target object area detector 117 can prevent the portionof the shadow from being included in the cropping image even if a shadowis generated due to the target object having a three-dimensional shape.Since the projection image projected from the projector 200 is notprojected in the shadow area, the imager 100 cannot calculate the heightinformation correctly. The target object area detector 117 can detectthe target object area with a high degree of precision by detecting thetarget object area so as not to include the shadow area, without usingthe height information calculated in the less reliable area.

It should be noted that the processing of step S501 may be omitted. Inthat case, the target object area detector 117 detects the target objectarea based on the height information of each pixel regardless of whetherthe shadow is illustrated in each pixel or not. Similarly, theprocessing in step S503 may also be omitted. In that case, the targetobject area detector 117 determines that the target object is notillustrated in the pixel if the height indicated by the heightinformation of the pixel is equal to or less than the predeterminedvalue. The order of processing in step S503 and step S505 may bereversed. In that case, when the difference between the color componentof each pixel and the color component of the background is less than thepredetermined value, the target object area detector 117 detects thetarget object area based on the height information of the pixel. Inthese cases, the target object area detector 117 can accurately detectthe target object.

The target object area detector 117 calculates the degree of the targetobject representing the likelihood that the target object is illustratedin each pixel, and the target object area detector 117 may determinewhether or not the target object is illustrated in each pixel dependingon whether or not the degree of likelihood of being the target object isequal to or more than a predetermined value. In that case, the targetobject area detector 117 calculates the degree of likelihood of beingthe target object, so that the degree of likelihood of being the targetobject is higher as the height illustrated in the height information ofeach pixel is higher, and the degree of likelihood of being the targetobject is lower as the height is lower. Further, the target object areadetector 117 calculates the degree of likelihood of being the targetobject, so that the degree of likelihood of being the target object ishigher as the difference between the color component of each pixel andthe color component of the background is larger, and the degree oflikelihood of being the target object is lower as the difference betweenthe color component and the color component of the background issmaller. As a result, the target object area detector 117 can detect thetarget object more accurately.

Alternatively, the target object area detector 117 may generate the edgeimage based on the height information of each pixel and detect thetarget object based on the edge image. In that case, for the heightindicated by the height information of each pixel in the projectioninput image, the target object area detector 117 calculates the absolutevalue (hereinafter referred to as adjacent difference value) of thedifference in the height between pixels on both sides in the horizontalor vertical direction. The target object area detector 117 extracts, asan edge pixel, a pixel whose adjacent difference value in the horizontalor vertical direction is more than a threshold value, and the targetobject area detector 117 generates an image composed of edge pixels asan edge image. The target object area detector 117 groups the edgepixels in the generated edge image by labeling, and the target objectarea detector 117 detects the area surrounded by the grouped edge pixelsas a target object area. In this case, the target object area detector117 can also accurately detect the target object.

Instead of detecting the target object area based on the color componentof each pixel in the non-projection input image, the target object areadetector 117 may detect the target object area based on the colorcomponent of each pixel in the projection input image. In this case, theestimated background area extractor 114 extracts the estimatedbackground area from the projection input image based on the heightinformation of each pixel in the projection input image. Based on thepixels in the estimated background area, the color component extractor115 determines the color component of the background in the projectioninput image, and extracts the color component of each pixel in theprojection input image. Then, the target object area detector 117detects the target object area based on the color component in thebackground and the color component of each pixel in the projection inputimage.

Through operation in accordance with the flowcharts illustrated in FIGS.5, 6, 9, 11, and 13, the image processing system 1 can easily andaccurately detect the target object area in the image acquired byimaging the target object. In particular, the image processing system 1uses the height information so that the image processing system 1 candetect the target object area with a high degree of accuracy even if thecolor of the entire or a part of the target object and the color of thebackground are similar to each other. The image processing system 1 candetect the target object area without using special background equipmentsuch as a blue back, special lighting equipment such as a backlight,etc.

REFERENCE SIGNS LIST

-   -   1 image processing system    -   100 imager    -   111 projection control module    -   112 input image obtaining module    -   113 height information calculator    -   114 estimated background area extractor    -   115 color component extractor    -   116 shadow area detector    -   117 target object area detector    -   118 cropping module    -   200 projector

What is claimed is:
 1. An image processing system comprising: aprojector for projecting a projection image having a stripe pattern or agrid pattern toward a target object; and an imager for capturing a firstinput image of the target object on which the projection image isprojected, and a second input image of the target object on which noprojection image is projected, wherein the imager includes: a processorfor calculating height information of the target object in the firstinput image by comparing the projection image with the first inputimage, extracting, from the second input image, an estimated backgroundarea not including the target object, based on the height information,determining a color component of the background area in the second inputimage, based on a pixel in the background area, and detecting a targetobject area where the target object exists in the second input image,based on the determined color component of the background area.
 2. Theimage processing system according to claim 1, wherein the projectorfurther projects light of a first color and light of a second colordifferent from the first color toward the target object, wherein theimager further captures a first color input image of the target objecton which the light of the first color is projected and a second colorinput image of the target object on which the light of the second coloris projected, and wherein the processor further determines, for eachpixel combination existing at a corresponding position in the firstcolor input image and the second color input image, whether or not adifference in a color component between pixels is equal to or less thana threshold value and whether or not the color component of each pixelis a black color, and detecting, as a shadow area, a position in thesecond input image corresponding to a position of a pixel combination ofwhich difference in a color component between pixels is equal to or lessthan a threshold value and of which the color component of each pixel isnot a black color, and detects the target object area so as not toinclude the shadow area detected by the shadow area detector.
 3. Theimage processing system according to claim 1, wherein the processor doesnot detect the shadow area when the determined color component of thebackground area is the black color.
 4. An image processing systemcomprising: a projector for projecting a projection image having astripe pattern or a grid pattern toward a target object; and an imagerfor capturing an input image of the target object, wherein the imagerincludes: a processor for calculating height information of the targetobject in the input image by comparing the projection image with theinput image and detecting a target object area where the target objectexists in the input image, based on the height information, wherein theprojector further projects light of a first color and light of a secondcolor different from the first color toward the target object, whereinthe imager further captures a first color input image of the targetobject on which the light of the first color is projected and a secondcolor input image of the target object on which the light of the secondcolor is projected, and wherein the processor further determines, foreach pixel combination existing at a corresponding position in the firstcolor input image and the second color input image, whether or not adifference in a color component between pixels is equal to or less thana threshold value and whether or not the color component of each pixelis a black color, and detecting, as a shadow area, a position in thesecond input image corresponding to a position of a pixel combination ofwhich difference in a color component between pixels is equal to or lessthan a threshold value and of which the color component of each pixel isnot a black color, and detects the target object area so as not toinclude the shadow area detected by the shadow area detector.
 5. Theimage processing system according to claim 4, wherein the processor doesnot detect the shadow area when the determined color component of thebackground area is the black color.
 6. An imager for capturing a firstinput image of a target object on which a projection image having astripe pattern or a grid pattern is projected and a second input imageof the target object on which no projection image is projected, theimager comprising: a processor for calculating height information of thetarget object in the first input image by comparing the projection imagewith the first input image, extracting, from the second input image, anestimated background area where it is estimated that a background isillustrated, based on the height information, wherein the backgrounddoes not include the target object, determining a color component ofbackground in the second input image, based on a pixel in the estimatedbackground area, and detecting a target object area where the targetobject exists in the second input image, based on the determined colorcomponent of the background.
 7. An object area detection method for animage processing system including a projector and an imager, the methodcomprising: projecting, by the projector, a projection image having astripe pattern or a grid pattern toward a target object; capturing, bythe imager, a first input image of the target object and a second inputimage of the target object on which no projection image is projected;calculating, by the imager, height information of the target object inthe first input image by comparing the projection image with the firstinput image; extracting, by the imager, from the second input image, anestimated background area where it is estimated that a background isillustrated, based on the height information, wherein the backgrounddoes not include the target object; determining, by the imager, a colorcomponent of background in the second input image, based on a pixel inthe estimated background area; and detecting, by the imager, a targetobject area where the target object exists in the second input image,based on the determined color component of the background.
 8. Acomputer-readable, non-transitory medium storing computer program,wherein the computer program causes an imager for capturing a firstinput image of a target object on which a projection image having astripe pattern or a grid pattern is projected by a projector and asecond input image of the target object on which no projection image isprojected to execute a process, the process comprising: calculatingheight information of the target object in the first input image bycomparing the projection image with the first input image; extracting,from the second input image, an estimated background area where it isestimated that a background is illustrated, based on the heightinformation, wherein the background does not include the target object;determining a color component of background in the second input image,based on a pixel in the estimated background area; and detecting atarget object area where the target object exists in the second inputimage, based on the determined color component of the background.